1) Field of the Invention
The present invention relates to a Joint Photographic Experts Group (JPEG) 2000 encoder that can encode motion pictures efficiently.
2) Description of the Related Art
The JPEG2000, which is standardized in January 2001 by the JPEG, being a joint organization of International Organization for Standardization (ISO) and International Telecommunications Union Telecommunication Standardization Sector (ITU-TS), is well known by the fact that it makes a block distortion in an image data with high compression ratio inconspicuous by replacing discrete cosine transform (DCT) employed by the conventional JPEG with discrete wavelet transform (DWT).
FIG. 1 is a schematic for explaining a functional configuration of a conventional JPEG2000 encoder (an image processing apparatus that encodes image data based on the JPEG2000). The image data subjected to pre-processing (such as DC level shift and component transform) is divided into a plurality of tiles, and the DWT is performed by a discrete wavelet transform (DWT) unit 400 for each tile.
An entropy coder 401 divides a set of coefficients of each tile obtained by the DWT into a plurality of code blocks, converts each code block into bit plane format, and encodes the bits in each bit plane in three passes of:
(1) significance propagation pass (SP);
(2) magnitude refinement pass (MRP); and
(3) clean up pass (CP),
from the most significant bit toward the least significant bit.
These passes have a function of classifying each bit in the code block corresponding to the contribution to the image quality. For example, (1) is a pass for selectively picking out a bit having large contribution to the image quality and encoding the bit. Likewise, (2) is a pass for encoding a bit having intermediate contribution and (3) is a pass for encoding a bit having small contribution. In coded data output from the entropy coder 401 to a coded data storage unit 402, bits of each code block are sorted in the order of from the higher bit plane to the lower bit plane, and in each bit plane, sorted in the order of from (1), (2), and (3).
Therefore, by truncating coded data from the end of each code block, the code amount can be adjusted to a predetermined value, while suppressing deterioration of the image quality (Rate-Distortion method). Control of the code amount is required particularly for motion pictures (the JPEG2000 is a standard targeting not only still pictures but also motion pictures) whose code amounts are apt to become large.
A rate controller 403 specifies number of passes for each code block, when code amount of one frame (one image) is a certain value. That is, a pass image quality value calculator 403a of the rate controller 403 calculates an image quality value of each pass coded by the entropy coder 401, and stores the value in a pass information storage unit 404 together with the number of bytes of each pass. When the processing has finished for all code blocks in one frame, a frame image quality value calculator 403b of the rate controller 403 calculates an image quality value of the frame when the code amount of the frame is set to a certain value (hereinafter, “target code amount”).
For this calculation, the graph shown in FIG. 3 is used conceptually. In the example shown in FIG. 3, a block distortion value is used as the image quality value. In this case, the larger the value, the worse the image quality is, and the smaller the value, the better the image quality is. In FIG. 3, the image quality value and the number of bytes at a position of pass 2 of code block 0, indicate the image quality value and the number of bytes of the code block 0 realized when passes up to pass 2 are left and pass 3 and after are truncated in the code block 0.
Therefore, when the image quality value of the currently processed frame (hereinafter, “current frame”) is provisionally set at a midpoint (in FIG. 3, “first provisional image quality value”) between the maximum image quality value (“image quality Max”) and the minimum image quality value (“image quality Min”), the first provisional image quality value will be achieved by leaving passes up to pass 3 in code block 0, up to pass 4 in code block 1, and up to pass 5 in code block 2. In this case, the code amount of one frame becomes the sum total of the code amounts of each code block in the frame, such as (the code amount of code block 0 in which passes only up to pass 3 are left)+(the code amount of code block 1 in which passes only up to pass 4 are left)+(the code amount of code block 2 in which passes only up to pass 5 are left).
When the code amount of one frame is not close to the target code amount, for example too much, the provisional image quality value is increased, for example, to a midpoint between the image quality Min and the first (previous) provisional image quality value (in FIG. 3, “second provisional image quality value”), to calculate the code amount again. As shown in FIG. 3, the code amount for one frame in this case becomes the sum total of (the code amount of code block 0 in which passes only up to pass 2 are left)+(the code amount of code block 1 in which passes only up to pass 3 are left)+(the code amount of code block 2 in which passes only up to pass 3 are left).
When the code amount is still not close to the target code amount, for example too small, the provisional image quality value is decreased, for example, to a midpoint between the first provisional image quality value and the second provisional image quality value (in FIG. 3, “third provisional image quality value”), to calculate the code amount again. Thus, by the binary searching method, the image quality value of the frame to make the code amount of the frame closest to the target code amount is calculated. Then, a truncation point in each code block to achieve the calculated image quality value is calculated.
When truncation points of all code blocks in one frame are determined, a file generating unit 405 in FIG. 1 generates a JPEG2000 file with only the number of passes informed for each code block from the frame image quality value calculator 403b of the rate controller 403. For example, when 2 is informed as the number of passes for code block 0, the coded data in pass 3 and thereafter in code block 0 are truncated.
FIG. 4 is a flowchart of a procedure in encoding processing of the image data by the conventional JPEG2000 encoder. The image data after pre-processing is transformed by the DWT unit 400 (step S701). The transformed data (coefficients) is then divided into code blocks by the entropy coder 401, and bits in each bit plane in each code block is encoded in each pass and stored in the coded data storage unit 402. At the same time, the pass image quality value calculator 403a of the rate controller 403 calculates the image quality value of each pass, and the calculated image quality value is stored in the pass information storage unit 404 together with the number of bytes of each pass (step S702).
After finishing the processing at step S702 for all passes in one code block (step S703: Yes), similar processing is performed for the next code block. After finishing the processing at steps S701 to S703 for all code blocks in one frame (step S704: Yes), the frame image quality value calculator 403b of the rate controller 403 performs rate control (step S705) to specify
the image quality value of the current frame to make the code amount of the frame closest to the target code amount; and
number of passes to be left in each code block to achieve the image quality value.
The file generating unit 405 then generates the JPEG2000 file (step S706) in which only the passes determined to be left at step S705 are incorporated.
FIG. 5 is a timing chart of the encoding processing of the image data in the conventional JPEG2000 encoder. The processing is roughly divided into three processing. In processing 1, the entropy coder 401 encodes each code block of one frame in each pass, and the rate controller 403 calculates the image quality value of each pass, and stores the image quality value in the pass information storage unit 404 together with the number of bytes of each pass. When the processing 1 has finished for one frame, the rate controller 403 calculates an image quality value which makes the code amount closest to the target code amount (processing 2), and also calculates number of passes to be left in each code block, and the file generating unit 405 generates the file based on the result thereof (processing 3).
Thus, according to the JPEG2000, a part of the image data (specifically, a bit having little influence to the image quality) is truncated, thereby adjusting the code amount to a predetermined value, while suppressing deterioration of the image quality. The conventional technology for suppressing the generated code amount includes the ones disclosed in Japanese Patent Application Laid-open Nos. 2001-258034, 2001-309381 and 2001-45485, other than the method described above.
However, in the conventional art explained above, the entropy coder 401 encodes not only passes that are not truncated but also passes that are eventually truncated by the file generating unit 405. Therefore, as the number of passes truncated in the latter processing increases, the former processing becomes inefficient.
For example, FIG. 6 is a graph of an example of how many passes are averagely truncated per one code block after encoding. For example, when the compression ratio is 100% (that is, no compression), in sample image 1, an average of 16.8 passes has been truncated per one code block. FIG. 7 is a graph in which each value in FIG. 6 converted into a ratio to a relevant value of when the compression ratio is 100%. As for sample image 3, for example, the number of passes to be truncated when the compression ratio is 10% is about half the number of passes to be truncated when the compression ratio is 100%. Thus, the number of passes to be truncated decreases as the image is compressed more compact. In other words, as the image is not compressed so much, more encoded passes are eventually truncated, which causes inefficiency of processing.